The present invention, in general relates to processes that normalize data and, more particularly, to systems that include a data base and software to calculate a percentile (i.e. a norm) that indicates job performance level (i.e., competency) of a given person""s file as compared with all other files in the data base.
There are countless situations that can benefit from the use of normalized data. By normalized data it is meant providing a percentile that indicates how a person, having data stored in a data base, compares with other people having data stored in the data base. This is discussed in greater detail hereinafter.
Usually, normalized data appertains to a measure of job performance or ability, often referred to as a competency. There is virtually no realm of human skills, preferences, and vocations that would not benefit from the availability of such data, as is also explained in greater detail hereinafter.
However, providing normalized data (i.e., norms) has been difficult to achieve heretobefore for a number of reasons. First, it is necessary to collect the data and to establish a file for each competency category that is being measured for each person for whom their performance is to be normed.
Second, it has been necessary to perform rather complex mathematical operations, including calculus, on the data to provide a norm. To avoid using calculus, an intermediate value, known as a xe2x80x9cZ scorexe2x80x9d, can be calculated.
The Z score is then used to refer to a lookup table (of Z scores) to determine the area under a curve which represents the actual normed data (i.e., percentile) for a given competency and therefore how that person""s scores compare in relation to all of the other people.
These steps must be repetitively performed for each person. Usually, the people doing this are not fluent with the mathematics or the calculus that is involved and so they may tend to shun the process.
Also, as new data becomes available for any of the people in the data base (i.e., having a file), it is necessary to redo the entire process so as to determine their most current percentile. As is discussed in greater detail hereinafter, it may be advantageous to use only recent data or it may be appropriate to use long-accumulated data to determine the norm.
The norm is expressed as a percentile. That percentile is indicative of job performance (i.e., competency) for a given area that is being measured when compared with other members in the group (i.e., having a file in the database).
It simply has not been practical heretobefore to often calculate norms. Neither has it been practical to collect the data in an efficient manner.
The inability to collect the data efficiently is compounded by the fact that the people themselves whose competencies are being measured may be scattered geographically.
For example, a firm may wish to determine the performance of its sales representatives. The sales representatives may be dispersed throughout the country or for that matter throughout the world.
Various factors may be included in the file in order to determine competency, such as the number of contacts each representative makes per month and the number of closes, (i.e., sales that result from each of those contacts). Many other factors may also be collected and deemed as useful in the norming process.
If this information is sent to an authorized company representative, for example an expert in human resources, there will be a delay in its acquisition. Accordingly, decisions based on the results of that data will of necessity be delayed until the data has been both collected and normed. As mentioned above, the norming itself is a cumbersome, slow process.
It is desirable to collect this information in as close to real-time as is possible and to do so in as cost-effective a manner as possible.
As mentioned above, many firms do not have the expertise xe2x80x9cin-housexe2x80x9d to even properly utilize this data (i.e., an ability to calculate the normed percentiles), nor do they have the means such as the necessary data base and software, none of which has been available heretobefore for a number of reasons.
As the mathematics involved is complex, it takes time to perform the necessary operations. There has been no way heretobefore to collect the data or to conveniently process the collected data (also known as raw data) to obtain the normed values.
One significant reason contributing to a lack of solution is that such software algorithms would, of necessity, be slow as they performed the complex mathematical operations. Therefore, it is desirable to be able to provide a quicker approach that can be used to normalize data. It simply has not been feasible previously.
If it were desired to access this data and initiate a calculation of the norms remotely, such as over the Internet, the slow speed could make such a system intolerable.
Ideally, if this data could be captured by a secure system connected to either the Internet or to an Intranet (i.e., an in-house computer system having remote access capabilities), then a means would be provided to capture the data in, or near, real-time.
Ideally, if this system included software that overcame the problems of slow calculations (which is compounded by the number of data points to be normed and the modem transfer rate), then an optimum system would be provided.
It is also important to note that access to such a system would likely be available for use on a fee schedule. That fee schedule could be based on a measurement of time that the database and system is accessed or it could be based on some other fee structure, such as the size of the database, or on a monthly fee. Obviously, a faster processing time will be of benefit to all concerned and would provide a more cost-effective solution.
Accordingly, subscribers could access the data and request normalized data for any of the files. If this could be done easily and as often as desired, then important decisions could be made quickly and efficiently. This would save a company a great deal of money. This is discussed in greater detail hereinafter.
In the above example, if one of the sales-persons was performing at a very low rate, closing a totally unacceptable rate of contacts and if that person had other poor parameters, knowing about this as soon as possible would be of great benefit to the company in making a decision to remove that person from a position which he is not well suited for and that was likely alienating many potential customers.
A normed percentile makes his performance or lack thereof obvious to those in a position to decide. This level of confidence is not available by examination of the xe2x80x9craw scoresxe2x80x9d, as is discussed in greater detail hereinafter.
Conversely, a person performing at a very high level should be soon rewarded lest he leave the firm for not being appropriately valued.
Some of the benefits the use of norms (i.e., normalized percentiles) can provide are as follows:
1. Norms provide a measure of competency based on the performance of any person relative to all of the members that are being compared.
2. Norms establish performance standards for the various groups (i.e., job classes).
3. Norms are useful in determining those members having exceptional ability in any group. This is useful to determine, among other things, which individuals are likely to become effective leaders or mentors.
4. Norms automate the evaluation and reporting of performance. This is a very difficult, time consuming, and subjective arena for most organizations.
5. Norms track and therefore help to determine the efficacy of programs that are designed to improve performance.
6. Norms provide valid, reliable data upon which human resource decisions are made. These include, among others, decisions that relate to hiring, firing, salary adjustments, bonuses, promotions, demotions, lateral changes in job assignments, etc.
7. Norms provide legally defensible data upon which these decisions are made. For example, a decision to terminate an employee can often subject an employer to liability arising from a claim of xe2x80x9cwrongful terminationxe2x80x9d. If the employer had a basis.of data upon which to justify that decision that was not subjective, then the decision by the employer to terminate would be defensible. Not only would this save the employer from false liability claims, but it would also allow the employer to more quickly weed out poorly performing employees.
Furthermore, when taken into account with item number xe2x80x9c5xe2x80x9d above, the normalized data would show how a poorly performing employee did not benefit from the opportunities to improve his performance that were afforded him. Not only that, but it would also demonstrate the efficacy of those improvement programs generally. This would only serve to strengthen the defensibility of any such decision to terminate (or to offer a transfer to a different location or assignment).
The only way to reliably measure, track, and improve performance on the job is by the use of numeric normative data. This is explained in greater detail hereinafter. For now it is important to consider some of the other advantages norming can provide.
Norms allow different competencies (with different numbers of xe2x80x9cattributesxe2x80x9d and xe2x80x9cbehaviorsxe2x80x9d) to be compared to one another on a common scale. The use of raw data cannot produce reliable performance standards. It is subjective and open to interpretation. Norms remove the subjectivity and great degree of interpretation that is commonly done by those who do the ratings (i.e., the raters) making performance assessments.
It is useful to examine what norms are in somewhat greater detail before proceeding.
Any new assessment instrument has no predetermined standards of xe2x80x9cpassingxe2x80x9d or xe2x80x9cfailingxe2x80x9d. For example, one may be looking for leadership qualities in an organization. This may be done so that the most appropriate people can be culled and trained for leadership positions. In order to do this an effective instrument to measure leadership must be provided.
This instrument begins in the form of a series of questions and the answers to those questions for any one particular person. This provides a rating. It is in the form of raw data and, initially, is of little use. Once it is converted to a norm, it is of great use.
For example, as the instrument is being created, communication skills are likely to be relevant to an assessment of leadership ability, whereas a favorite dessert item is far less likely to have relevance.
Consequently, performance standards must be determined and evaluated on the basis of empirical data and through the calculation of norms. Over time, the reliability of the performance standards is both proved and improved.
Norms compare an individual""s score to the scores obtained by everyone else that is being rated.
Norms set a clear empirical""standard for what is considered to be performance that is below expectations, meets expectations, and that which exceeds expectations.
Raw scores on any assessment instrument are essentially meaningless. For example, to say that a person has a total raw score of 14 on the xe2x80x9ccritical thinkingxe2x80x9d competency assessment conveys little or no information about his or her standing in this area.
Is a raw score of 14 good or bad? Is that person""s performance below, the same as, or above the performance of his or her peers? Should that person receive a large salary increase for exceptional performance or no increase and a warning for very poor performance? Clearly, a raw score is essentially a meaningless number. This is where subjective evaluation can enter. If the rater xe2x80x9clikesxe2x80x9d the person, then a raw score of 14 may be considered good. If the rater xe2x80x9cdislikesxe2x80x9d the person, then the same raw score may be subjectively used in support of an assessment of poor performance or competency.
In order to determine more precisely an individual""s exact level of performance, the raw scores must be converted into a standardized score for any particular area of interest. That standardized score is a norm score that is expressed relative to the population mean and standard deviation of the competency that is in question.
Accordingly, norms indicate an individual""s relative standing to the population, thus permitting an evaluation of his or her performance in reference to other people.
Norms also provide comparable measures that permit a direct comparison of the individual""s performance across different competencies (for example, critical thinking, communication, etc.) and across different job classes (Job 1, Job 2, etc.). Without norms, these comparisons simply cannot be made.
Each competency raw score in a performance appraisal may include a different number of items and may be measuring a different set of behaviors and attributes. All raw competency scores are transformed to a standardized distribution with a mean of 0.0 and a standard deviation of 1.0. These standardized (or normed) values can thus all be expressed as cumulative percentages (from 0% to 100%) thereby allowing for an xe2x80x9capples to applesxe2x80x9d comparison across each competency and within each job classification.
For example, assume that a person, xe2x80x9cAssociate 1xe2x80x9d had a raw score of 16 on both xe2x80x9ccritical thinkingxe2x80x9d and xe2x80x9ccommunicationxe2x80x9d. Let""s also assume the distribution of raw xe2x80x9ccritical thinkingxe2x80x9d scores had a mean of 15.0 and a standard deviation of 2.0, and that the distribution of raw xe2x80x9ccommunicationxe2x80x9d scores had a mean of 15.0 (the same mean) but a standard deviation of 1.0. In this case, his normed critical thinking score would be 69% and his normed communication score would be 84%.
In other words, he would be in the 69th percentile of critical thinking, as compared with other associates, and he would be in the 84th percentile for communication. Accordingly, it would be known that he had moderately good critical thinking ability and very good communication ability when compared with his peers. By looking at the scores of other associates, an effective assessment of his competency is provided.
This also serves as a concrete example of how two identical raw scores for different areas of competency could, in reality, indicate drastically different levels of performance. Looking at the raw scores one would think that Associate 1 was equally as competent in his critical thinking abilities as he was in his communication abilities. The norms show us that this is simply not true. His performance must be mathematically compared with that of his peers in order to determine his true level of competency.
If any of the values (means, standard deviations) were to change, and they all will change over time as new performance data is obtained, then the outcome would again produce very different definitive assessments of competency. This, in turn, indicates the need for not only normalizing data, but of doing so in a continuous fashion.
It has clearly been shown the value of normalized data. It has also been shown the need to obtain this information quickly, and from a wide variety of locations geographically. An ability to collect raw data scores and store them in a data base and calculate the norms on demand would also provide the following benefits:
1. Performance standards would always be current. With an effective auto-norming capability, norms could be determined xe2x80x9con the flyxe2x80x9d as new data is collected. As the new data is entered, those authorized to access that data (typically human resource personnel) would be able to determine performance as often as desired and in the competencies that are of interest.
2. Additionally, if desired, the database could be either purged of old data or considered in part so that the norms could be calculated based only upon the most recent data, thereby providing a mechanism to track improved (i.e., changing) competencies in both an accurate and a timely way.
There is great flexibility provided in this regard. For example, norms can be calculated on all the data collected to date, or by using data from specific evaluation periods. This allows for human resource personnel to make decisions to either maintain or to raise a xe2x80x9cperformance barxe2x80x9d, as needed.
Over time, the net result will be to increase the performance of all of the members of the group (i.e., Job 1, Job 2, etc.). This will result in raising the performance xe2x80x9cbarxe2x80x9d for each job description as the competencies in that group generally improve.
Obviously, the organization that uses this approach will improve their ability to provide good service and excellent products to their customers. Therefore, the organization will benefit greatly from the use of norms as they evaluate the most important resource that they have, their employees.
Accordingly, there exists today a need for an auto-norming process and system.
2. Description of Prior Art
Norms are, in general, known. For example, the following patent describes one such types of a system:
U.S. Pat. No. 4,545,388 to John E. Roy, Oct. 8, 1985.
While the structural arrangements of the above described system, at first appearance, has similarities with the present invention, it differs in material respects. These differences, which will be described in more detail hereinafter, are essential for the effective use of the invention and which admit of the advantages that are not available with the prior system.
It is an object of the present invention to provide an auto-norming process and system for the norming of raw data.
It is also an important object of the invention to provide an auto-norming process and system for the norming of data on demand.
Another object of the invention is to provide an auto-norming process and system to provide an auto-normed percentile that is indicative of the level of a competency.
Still another object of the invention is to provide an auto-norming process and system for norming of data that includes a data base accessible by the Internet.
Still another important object of the invention is to provide an auto-norming process and system for norming of data that includes a data base and processing capability that are accessible by an Intranet (i.e., a local area network or LAN).
One further additional object of the invention is to provide an auto-norming process and system that recalculates a norm for a competency when data in the database changes.
One more additional important object of the invention is to provide an auto-norming process and system that relies upon an algebraic computation to approximate a normed score.
Still yet another object of the invention is to provide an auto-norming process and system that is available for use for the payment of a fee.
Yet another important object of the invention is to provide an auto-norming process and system that is cost-effective.
Still yet another important object of the invention is to provide an auto-norming process and system that is reliable.
Still yet one other important object of the invention is to provide an auto-norming process and system that is able to accommodate any number of people and any number of items (i.e., competencies) that are to be measured.
Still yet one other especially important object of the invention is to provide an auto-norming process and system that is able to vary the number of people and the number of items (i.e., competencies) that are to be measured.
Briefly, an auto-norming process and system that is constructed in accordance with the principles of the present invention has a database that includes a file for each person in a group. Each file includes a rating for each item of interest for that person. The files are updated as new data for each item becomes available. Processing capability is provided to calculate the norm for each competency (i.e., group of items) for each person. That norm is provided to authorized personnel as a percentile that is indicative of that person""s competency for each item when compared with the competencies of other members in the group.